-
Notifications
You must be signed in to change notification settings - Fork 193
Add Example for Comet #305
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
requirements.txt
|
@ParagEkbote Thanks for the PR. I'm worrying about the CI failure. All examples in this repository are continuously tested by the GitHub Actions, so I expect that this example can be tested too. It seems that the API key of the Comet is missing. Is is possible to generate the API key which can be used here? |
No, we will probably need to add it as a GitHub repo secret in Settings->Secrets and Variables->Actions. WDYT? cc: @HideakiImamura |
|
Thanks for the reply. I understood. Is it OK to use the generated API key of one of us (comitters) or should be use any universal API key? |
Unless there is an enterprise plan available, we will need to use an individual API key. cc: @HideakiImamura |
|
I'm not sure but how about https://www.comet.com/docs/v2/api-and-sdk/python-sdk/advanced/running-offline/ like WandB example
|
|
I discussed with other committers. It would be great to remove the CI configuration file from this PR. |
I have removed the CI config file. Could you please review? cc: @HideakiImamura |
Co-authored-by: Kento Nozawa <[email protected]>
nzw0301
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about using the same objective and printing result code as
https://github.com/optuna/optuna-examples/blob/main/wandb/wandb_integration.py?
This is because it would be more easily understood without manually calling log_metric and similar code to other MLops code (namely, wandb here).
|
Hi @HideakiImamura, thank you for directing us. |
|
I have updated the example. Could you please review? cc: @nzw0301 |
|
Lovely, I will check the changes this weekend! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi, I think the current script does not work correctly at
comet_callback = CometCallback(project_name="comet-optuna-sklearn-example")Can you fix this?
Co-authored-by: Kento Nozawa <[email protected]>
|
I apologize for making breaking changes in my example. I've corrected it now and it shows the following Output: Number of finished trials: 20 Best trial: Params: Could you please review? cc: @nzw0301 |
nzw0301
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lovely. Could you rename the python script from comet_callback.py to comet_integration.py?
I realised that offline evaluation without an API key might be impossible, so please ignore my careless comments. Sorry. |
|
I've renamed the file. Could you please review? cc: @nzw0301 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry minor things.
Co-authored-by: Kento Nozawa <[email protected]>
|
Are any pending changes that need to be addressed? Could you please let me know? |
|
This pull request has not seen any recent activity. |
|
Let me merge this PR since this PR already got approval from the maintainer. @ParagEkbote Thank you for your contribution! |

Fixes #269
I've setup an example with the scikit-learn breast cancer dataset and a Random Forest Classifier model. How do I set up the comet API key for CI?
cc: @not522